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Abstract 

For every total recursive time bound t, a constant fraction of all compressible (low 
Kolmogorov complexity) strings is t-bounded incompressible (high time-bounded 
Kolmogorov complexity); there are uncountably many infinite sequences of which 
every initial segment of length n is compressible to logn yet t-bounded incom- 
pressible below |n — logn; and there are a countably infinite number of recursive 
infinite sequences of which every initial segment is similarly t-bounded incompress- 
ible. These results and their proofs are related to, but different from, Barzdins's 
lemma. 
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1 Introduction 



Informally, the Kolmogorov complexity of a finite binary string is the length 
of the shortest string from which the original can be losslessly reconstructed 
by an effective general-purpose computer such as a particular universal Turing 
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machine U . Hence it constitutes a lower bound on how far a lossless compres- 
sion program can compress. Formally, the conditional Kolmogorov complexity 
C{x\y) is the length of the shortest input z such that the universal Turing 
machine U on input z with auxiliary information y outputs x. The uncondi- 
tional Kolmogorov complexity C{x) is defined by C{x\e) where e is the empty 
string (of length 0). Let t be a total recursive function. Then, the time-bounded 
conditional Kolmogorov complexity C^{x\y) is the length of the shortest input 
z such that the universal Turing machine U on input z with auxiliary in- 
formation y outputs X within t{n) steps where n is the length in bits of x. 
The time-bounded unconditional Kolmogorov complexity C*(x) is defined by 
C*(x|e). For an introduction to the definitions and notions of Kolmogorov 
complexity (algorithmic information theory) see [3]. 



1.1 Related Work 



Already in 1968 J. Barzdins [2] obtained a result known as Barzdins's lemma, 
probably the first result in resource-bounded Kolmogorov complexity, of which 
the lemma below quotes the items that are relevant here. Let x denote the 
characteristic sequence of an arbitrary recursively enumerable (r.e.) subset A 
of the natural numbers. That is, x is an infinite sequence X1X2 • • • where bit 
Xi equals 1 if and only ii i & A. Let Xi-.n denote the first n bits of x, and 
let C{xi.n\n) denote the conditional Kolmogorov complexity of xi:n, given the 
number n. 

Lemma 1 (i) For every characteristic sequence x of a r.e. set A there exists 
a constant c such that for all n we have C{xi:n\n) < logn + c. 

(ii) There exists a r.e. set A with characteristic sequence x such that for every 
total recursive function t there is a constant Ct with < q < 1 such that for 
all n we have C^{xi.n\n) > ctu. 

Barzdins actually proved this statement in terms of D.W. Loveland's version 
of Kolmogorov complexity [4j, which is a slightly different setting. He also 
proved that there is a r.e. set such that its characteristic sequence x = X1X2 ■ ■ ■ 
satisfies C{xi:n) > logn for every n. Kummer [5], Theorem 3.1, solving the 
open problem in Exercise 2.59 of the first edition of [3] proved that there 
exists a r.e. set such that its characteristic sequence ( = (1,(2, ■■■ satisfies 
C(Ci:n) > 2 logn — c for some constant c and infinitely many n. 

The converse of item (i) does not hold. To see this, consider a sequence x = 
X1X2 ■ ■ ■ and a constant c' > 2, such that for every n we have C{xi:n\n) > 
n — c'logn By item (i), x can not be the characteristic sequence of a r.e. set. 
Transform x into a new sequence ( = Xi<^iX2Q^2 ■ ■ ■ with = a string of 
Os of length 2\ While obviously ( can not be the characteristic sequence of a 
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r.e. set, there is a constant c such that for every n we have that C{C,i-n\n) < 
log n + c. 

Item (i) is easy to prove and item (ii) is hard to prove. Putting items (i) 
and (ii) together, there is a characteristic sequence x of a r.e. set A whose 
initial segments are both logarithmic compressible and time-bounded lin- 
early incompressible, for every total recursive time bound. Below, we iden- 
tify the natural numbers with finite binary strings according to the pairing 
(e, 0), (0, 1), (1, 2), (00, 3), (01, 4), . . . , where e again denotes the empty string. 



1.2 Present Results 



Theorem 1 Let ko, ki be positive integer constants and t a total recursive 
function. 

(i) A constant fraction of all strings x of length n with C{x\n) < kologn 
satisfies C'^{x\n) > n — ki. {Lemma\^. 

(ii) Lett{n) > cn fore > 1 sufficiently large. A constant fraction of all strings 
X of length n with C{x\n) < kologn satisfies C^{x\n) < kologn (Lemma 0). 

(iii) There exist uncountably many {actually 2^°) infinite binary sequences uo 
such that C{uJi-n\n) < \ogn and C^{ijJi-n\n) > ^n — log n for every n; moreover, 
there exist a countably infinite number of {that is Kq) recursive infinite binary 
sequences u {hence C{ijJi-n\n) = 0(1)) such that C^{uJi:n\n) > — logn for 
every n {Lemma\^. 

Note that the order of quantification in Barzdins's lemma is "there exists a 
r.e. set such that for every total recursive function t there exists a constant 
Cf." In contrast, in item (iii) we prove "there is a positive constant such that 
for every total recursive function t there is a sequence u;." While Barzdins's 
lemma proves the existence of a single characteristic sequence of a r.e. set 
that is time-limited linearly incompressible, in item (iii) we prove the exis- 
tence of uncountably many sequences that are logarithmically compressible 
over the initial segments, and the existence of a countably infinite number 
of recursive sequences, such that all those sequences are time-limited linearly 
incompressible. 

We generalize item (i) in Corollaries [T] and [2l Section [2] presents preliminaries. 
Section [3] gives the results on finite strings. Section |4] gives the results on 
infinite sequences. Finally, conclusions are presented in Section [51 The proofs 
for the results are different from Barzdins's proofs. 
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2 Preliminciries 



A (binary) program is a concatenation of instructions, and an instruction is 
merely a string. Hence, we may view a program as a string. A program and 

a Turing machine (or machine for short) are used synonymously. The length 
in bits of a string x is denoted by \x\. If m is a natural number, then |m| is 
the length in bits of the mth binary string in length-increasing lexicographic 
order, starting with the empty string e. We also use the notation IjS"! to denote 
the cardinahty of a set S. 

Consider a standard enumeration of all Turing machines Ti, T2, .... Let U 
denote a universal Turing machine such that for every y G {0, 1}* and i > I 
we have U{i,y) — Ti{y). That is, for all finite binary strings y and every 
machine index i > 1, we have that ?7's execution on inputs i and y results 
in the same output as that obtained by executing Tj on input y. Let t be a 
total recursive function. Fix U and define that C{x\y) equals min{|p| : p G 
{0,1}* and U{p,y) = x}. For the same fixed U, define that C^{x\y) equals 
min{|p| : p G {0, 1}* and U{p,y) — x in t{\x\) steps}. (By definition the sets 
over which is minimized are countable and not empty). 



3 Finite Strings 

Lemma 2 Let ko, ki be positive integer constants and t be a total recursive 
function. There is a positive constant Ct such that for sufficiently large n the 
strings x of length n satisfying C*(x|n) > n — ki form a Cf-fraction of the 
strings y of length n satisfying C(t/|n) < k^logn. 

Proof. The proof is by diagonalization. We use the following algorithm with 
inputs t,n,ki and a natural number m. 

Algorithm A{t, n, ki, m) 

Step 1. Using the universal reference Turing machine U, recursively enumer- 
ate a finite list of all binary programs p of length \p\ < n — ki. There are at 
most 2"/2'^'^ — 1 such programs. Execute each of these programs on input n. 
Consider the set of all programs that halt within t{n) steps and which output 
precisely n bits. Call the set of these outputs B. Note that \B\ < 2"/2*^i — 1 
and it can be computed in time 0{2'^t{n) /2^^). 

Step 2. Output the (m + l)th string of length n, say in the lexicographic 
order of all strings in {0, 1}" \ i? and halt. If there is no such string then halt 
with output ±. End of Algorithm 
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Because of the selection process in Step 1, |{0, 1}" \ i?| > 2" — 2^/2^^ + 1 and 
every xG{0,l}"\i? has time-bounded complexity 

C\x\n)>n~h. (1) 



For \m\ < k^logn — c, where the constant c is defined below, and provided 
{0, 1}" \ i? is sufficiently large, that is, 

n'V2^ < 2" (l - ^) + 1, (2) 



there are at least /2'^ strings x of length n that will be output by the 
algorithm. Call this set D. Each string x G -D satisfies 

C(x|t, n, /ci, A^p) < \m\ < ko log n — c. (3) 



Since we can describe the fixed t, ko, ki, A, a program p to reconstruct x from 
these data, and the means to tell them apart, in an additional constant num- 
ber of bits, say c bits (in this way the quantity c can be deduced from the 
conditional), it follows that C{x\n) < kologn. For given ko, ki, and c, inequal- 
ity ([2]) holds for every sufficiently large n. For such sufficiently large n, the 
cardinality of the set of strings of length n satisfying both C{x\n) < kologn 
and C*(x|n) > n — fci is at least |D| = n'^"/2'^. Since the number of strings x of 
length n satisfying C{x\n) < kologn is at most X]f=o'^"2* < 2n'"\ the lemma 
follows with Q = 1/2'=+^ □ 



Corollary 1 Let ko be a positive integer constant and t be a total recursive 
function. For every sufficiently large natural number n, the set of strings x 
of length n such that C^{x\n) ^ kologn is a positive constant fraction of the 
strings y of length n satisfying C{y\n) < kologn. 

We can generalize Lemma [21 Let t be a total recursive function, and /, g be 
total recursive functions such that (jl]) below is satisfied. 

Corollary 2 For every sufficiently large natural number n, the set of strings 
X of length n that satisfy both C{x\n) < f{n) and C*(a;|?7,) > g{n) is a positive 
constant fraction of the strings y of length n satisfying C{y\n) < f{n). 

Proof. Use a similar algorithm A{t,n, g,m) with \p\ < g{n) in Step 1, and 
|"^| ^ /('^) — c in the analysis. Require 

2/(")-<: < 2" - 23(") + 1. (4) 



□ 
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Lemma 3 Let t be a total recursive function with t{n) > cn for some c > 1 and 
fco be a positive integer constant. For every sufficiently large natural number 
n, there is a positive constant Ct such that the set of strings x of length n 
satisfying C*(x|n) < kologn is a effraction of the set of strings y of length n 
satisfying C{y\n) < fco log n. 

Proof. We use the following algorithm that takes positive integers n,m as 
inputs and computes a string x of length n satisfying C*(x|n) < kologn — c. 

Algorithm B{n,m) 

Output the string 0"'^'™'+^'(m + 1) (where |m + 1| is the length of the string 
representation of m + 1) and halt. End of Algorithm 

Let ko be a postive integer and c a positive integer constant chosen be- 
low. Consider strings x that are output by algorithm B and that satisfy 
C^{x\n, B,p) < \m\ < k^logn — c with c the number of bits to contain de- 
scriptions of B and k^, a program p to reconstruct x from these data, and the 
means to tell the constituent items apart. Hence, C^{x\n) < ko logra. The run- 
ning time of algorithm B is t{n) = 0{n), since the output strings are length n 
and to output the mth string with m < 2'^o'°s"-'= we simply take the binary 
representation of m and pad it with nonsignificant Os to length n. Obviously, 
the strings that satisfy C*(x|n) < ko \ogn are a subset of the strings that sat- 
isfy C{x\n) < ko logn. There are at least n^" /2'^ strings of the first kind while 
there are at most 2n'^° strings of the second kind. Setting q = 1/2'^+^ finishes 
the proof. □ 

It is well known that if we flip a fair coin n times, that is, given n random bits, 
then we obtain a string x of length n with Kolmogorov complexity C{x\n) > 
n — c with probability at least 1 — 2~'^. Such a string x is algorithmically 
random. We can also get by with less random bits to obtain resource-bounded 
algorithmic randomness from compressible strings. 

Lemma 4 Let a, b be constants as in the proof below. Given the set of strings 
X of length n satisfying C{x\n) < k^logn, a total recursive function t, the 
constant ki as before, and 0{ab\ogn) fair coin flips, we obtain a set of 0{ab) 
strings of length n such that with probability at least 1 — 1/2^ one string x in 
this set satisfies C*(x|n) > n — ki. 

Proof. By Lemma [21 a Qth fraction of the set A of strings x of length n that 
have C{x\n) < fco logn also have C*(x|n) > n — ki. Therefore, by choosing, 
uniformly at random, a constant number a of strings from the set A we increase 
(e.g. by means of a Chernoff bound [3]) the probability that (at least) one of 
those strings cannot be compressed below n — fci in time t{n) to at least |. To 
choose any one string from A requires O(logn) random bits by dividing A in 
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two equal size parts and repeating this with the chosen half, and so on. The 
selected a elements take O(alogn) random bits. Applying the previous step 
b times, the probability that at least one of the ab chosen strings cannot be 
compressed below n — ki bits in time t{n) is at least 1 — 1/2*. □ 



4 Prom Finite Strings to Infinite Sequences 

We prove a result reminiscent of Barzdins's lemma. Lemma [H In Barzdins's 
version, characteristic sequences u of r.e. sets are considered which by Lemma[T] 
have complexity C(a;i.„|n) < logn + c. Here, we consider a wider class of se- 
quences of which the initial segments are logarithmically compressible (such 
sequences are not necessarily characteristic sequences of r.e. sets as explained 
in Section [LT]). 

Lemma 5 Let t be a total recursive function, (i) There are uncountably many 
{actually 2^") sequences u = ujiuj2 ■ ■ ■ such that both C{uji.n\n) < logn and 
C*{ijJi-n\n) >\n — logn for every n. 

(ii) The set in item (i) contains a countably infinite number of [that is Kg) 
recursive sequences ui = ujiuj2 ■ ■ ■ such that C*(u;i;„|n) > \n — logn for every 
n. 

Proof, (i) Let g{n) = \n — logn. Let c > 2 be a constant to be chosen later, 
mi = c2\ B{i), C(i), D{i) C {0, 1}*"' for z = 0, 1, . . ., and = {e}. The 

C sets are constructed so that they contain the target strings in the form of a 
binary tree, where C{i) contains all target strings of length mj. The B{i) sets 
correspond to forbidden prefixes of length m^. The D{i) sets consist of the set 
of strings of length with prefixes in C(i — 1) from which the strings in C{i) 
are selected. 

Algorithm C{t,g): 

for i := 0, 1, . . . do 

Step 1. Using the universal reference Turing machine U, recursively enumer- 
ate the finite list of all binary programs p of length \p\ < g{mi) with mi = c2* 
and the constant c defined below. There are at most 

29{m,) - 1 such programs. 

Execute each of these programs on all inputs rrij + j with < j < mj. Con- 
sider the set of all programs with input + j that halt with output x = yz 
within t{\x\) time with |x| = m^ + j, y E C{i — 1) (then \y\ = mj_i for 
i > and \y\ = for i = 0), and z is a binary string such that x satisfies 
iTT'i < l^^l < f^i+i- There are at most mj(2^*^™''' — 1) such x's. Let B{i) be the 
set of the rrij-length prefixes of these x's. Then, |-B(i)| < mj(2^*^™''' — 1) and it 
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can be computed in time O (mi2^^'^^H{mi+i)) . Note that if m e {0, 1}"^' \ B(i) 
then C^{uw\ \uw\) > g{\u\) for every w such that \uw\ < rrii+i. 

Step2. LetC(i-l) = {xi, X2, . . . , Xh} and D{i) = {C{i-l){0,l}* n{OA}'^')\ 
B{i). for / := 1, . . . , /i do for := 0, 1 do put the fcth string with initial 
segment x;, in the lexicographic order of D{i), in C{i). If there is no such 
string then halt with output ±. od od od End of Algorithm 

Clearly, C(i){0, 1}* C C{i - 1){0, 1}* for every i = 0, 1, . . . . Therefore, if 

oo 

nc(z){o,ir ^0, (5) 

1=0 

then the elements of this intersection constitute the infinite sequences uj in the 
statement of the lemma. 

Claim 1 With g{'mi) = ^rrii — logmj, we have \C{i)\ = 2*+^ for i = 0, 1, 

Proof. The proof is by induction. Recall that rrii = cT with the constant c > 2. 
Bast case: |C(0)| = 2 since C(-l) = {e} and |D(0)| > 2"^" - mo(2^'(™") - 1) > 
2. 

Induction: Assume that the lemma is true for every < j < i. Then, every 
string in C(z — 1) has two extensions in C(z), since for every string in C(i — 1) 
there are 2™''~''"»-i extensions available of which at most |-B(«)| < mj(2^*^'^') — 1) 
are forbidden. Namely, 2'"^-'"-i - \B{i)\ > T^^l'^ - 29('"0+i°g'^> + > 2. 
Hence it follows that the binary /c-choice can always be made in Step 2 of the 
algorithm for every /. Therefore |C(z)| = 2*"*"^. □ 

Let a constant c\ account for the constant number of bits to specify the func- 
tions t, (7, the algorithm C, and a reconstruction program that executes the 
following: We can specify every initial mj-length segment of a particular uj in 
the set on the lefthand side of ([5]) by running the algorithm C using the data 
represented by the C\ bits, m^, and the indexes kj G {0, 1} of the strings in 
with initial segment in C(j — 1), < j < i, that form a prefix of lo. 
Therefore, 

C(u;i:mjmj) < ci + i + 1. 

Setting c = 2'^^+^ yields Ciuj^.m^rni) < logc + z = logmj. By the choice of B{i) 
in the algorithm we know that C\LJi,rni+j\TTT'i+j) > 5'('^j) for every j satisfying 
< j < rrii. Because 2mj = mj+i, for every n satisfying rrii < n < mj+i, 
C^{uJi:n\n) > ^rrii — log rrii > ^n — ^ogn. Since this holds for every i = 0,1, . . . , 
item (i) is proven with C*(u;i:„|n) > \n — logn for every n. The number of 
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cij's concerned equals the number of paths in an infinite complete binary tree, 
that is, 2^". 

(ii) This is the same as item (i) except that we always take, for example, 
ki = (no binary choice) in Step 2 of the algorithm. In fact, we can specify 
an arbitrary computable 0-1 valued function to choose the fcj's. There are a 
countably infinite number of (that is Kq) such functions. The specification of 
every such function takes C{(f)) bits. Hence we do not have to specify the 
successive ki bits, and C{LJi,n\n) = ci + 1 + C(0) = 0(1) with ci the constant 
in the proof of item (i). Trivially, still C*(c(Ji:mi+j|^j + j) > fi'(^j) for every 
j satisfying < j < mj. Since this holds for every i = 0, 1, ... , item (ii) is 
proven by item (i). □ 



5 Conclusions 

We have proved the items promised in the abstract. In Lemma [5] we iter- 
ated the proof method of Lemma [2] to prove a result which is reminiscent 
of Barzdins's lemma [H relating compressiblity and time-bounded incompress- 
iblity of infinite sequences in another manner. Alternatively, we could have 
studied space-bounded incompressibility. It is easily verified that the results 
also hold when the time-bound t is replaced by a space bound s and the time- 
bounded Kolmogorov complexity is replaced by space-bounded Kolmogorov 
complexity. 
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